Near Shannon Limit Performance of Low Density Parity Check Codes

We report the empirical performance of Gallager’s low density parity check codes on Gaussian channels. We show that performance substantially better than that of standard convolutional and concatenated codes can be achieved; indeed the performance is almost as close to the Shannon limit as that of Turbo codes. A linear code may be described in terms of a generator matrix G or in terms of a parity check matrix H, which satisfies Hx = 0 for all codewords x. In 1962, Gallager reported work on binary codes defined in terms of low density parity check matrices (abbreviated ‘GL codes’) [5, 6]. The matrix H was defined in a non-systematic form; each column of H had a small weight (e.g., 3) and the weight per row was also uniform; the matrix H was constructed at random subject to these constraints. Gallager proved distance properties of these codes and described a probability-based decoding algorithm with promising empirical performance. However it appears that GL codes have been generally forgotten, the assumption perhaps being that concatenated codes [4] were superior for practical purposes (R.G. Gallager, personal communication). During our work on MN codes [8] we realised that it is possible to create ‘good’ codes from very sparse random matrices, and to decode them (even beyond their minimum distance) using approximate probabilistic algorithms. We eventually reinvented Gallager’s decoding algorithm and GL codes. In this paper we report the empirical performance of these codes on Gaussian channels. We have proved theoretical properties of GL codes (essentially, that the channel coding theorem holds for them) elsewhere [9]. GL codes can also be defined over GF (q). We are currently implementing this generalization. We created sparse random parity check matrices in the following ways. Construction 1A. An M by N matrix (M rows, N columns) is created at random with weight per column t (e.g., t = 3), and weight per row as uniform as possible, and overlap between any two columns no greater than 1. (The weight of a column is the number of non-zero elements; the overlap between two columns is their inner product.)